This page describes in detail the action requests that the robot controller can send. It also describes the response-receiving procedures for the asynchronous requests. The page is valid for the Communication interface BPS 1.12 and LS 1.5.
0 Request overview
Each action request has a request ID. Most IDs are the same in Bin Picking Studio (BPS) and Locator Studio (LS). Some requests that exist in both products have different IDs. The table below lists all requests of the current Communication interface.
The Robot module handles the request IDs internally. The robot program calls the procedures of the Photoneo robotic API and never sends the ID directly. The IDs are listed here because they appear in logs, in the Studio console and in some integration guides.
Values returned by the requests (other than error codes, poses and trajectories) are sent as Info data.
1 Bin picking action requests
Note: Most bin picking requests are bound to a specific vision system, so the vision system ID is a required parameter. The exception is the Change scene state request, which requires the scene state ID.
1.1 Initialization request
Required running process
Deployment
Introduced in the Communication interface version
BPS 1.0.0
Required parameters
joint values of the Start pose, joint values of the End pose, vision system ID
Description
Initializes a vision system. The joint values of the Start pose and End pose defined in the robot controller are sent to the vision controller. The vision controller uses them as the start and end points of the bin picking routine for the requested vision system.
It blocks the robot program only for a very short time.
Usage
Typically, send it only once at the start of the robot program, when the deployment is already running. After that, send it again only if:
- the Start pose or End pose must change
- the deployment was stopped and started again
- the localization memory must be cleared (if the localization memory clearing parameter is turned on)
1.2 Scan request
Required running process
Deployment
Introduced in the Communication interface version
BPS 1.0.0
Required parameters
vision system ID
Description
The sensor of the requested vision system triggers a new scan (full/fast). Immediately after the scan is finished, the localization starts to report found objects, and path planning calculates trajectories for them.
For hand-eye vision systems, the robot pose at the moment of the scan is taken from the Robot State Server. The request itself only contains the vision system ID. The Robot State Server must be running.
The scan can take several seconds. Its result is sent when the scan is finished. To not block the robot program during this time, the response is received in a separate procedure, Wait for scan completion. Always call this procedure after a scan request.
Usage
Hand-eye vision system
Typically, send it in each cycle as soon as the robot finishes a pick (bin picking routine) and is still above the bin (single-bin application). This gives BPS the most time for localization and path planning before the robot is ready to pick the next object.
Call the Wait for scan completion procedure immediately after the Scan request. It blocks the robot program until the scan is finished (the robot must not move during the scan).
Extrinsic vision systems
Typically, send it in each cycle as soon as the robot finishes a pick (bin picking routine) and leaves the scanning volume. This gives BPS the most time for localization and path planning before the robot is ready to pick the next object.
Then call the Wait for scan completion procedure to receive the result of the scan.
1.3 Trajectory request
Required running process
Deployment
Introduced in the Communication interface version
BPS 1.0.0
Required parameters
vision system ID
(timeout - set in the solution settings)
Description
Requests the vision controller to send a bin picking trajectory and other bin picking data (gripper commands, grasping data,
Info data) to the robot controller.
The vision controller chooses the most suitable object from the objects with successfully computed trajectories. The choice is based on the configured priority settings at the moment the request is received. If no such object exists yet, the vision controller chooses the first object that gets a successfully computed trajectory, or it sends an error to the robot controller.
The Trajectory request does not block the robot program, because the bin picking trajectory is received by a separate procedure, Receive trajectory. Always call this procedure after a trajectory request.
Usage
Normally, send it only once in each cycle, after a scan request for the same vision system. A trajectory request for a different vision system than the last scan request causes an error.
It is good practice to send the trajectory request just before the bin picking routine starts (when the robot is at the Start pose or on its way there). This gives the vision controller as many pick candidates as possible.
Then receive the trajectory with the Receive trajectory procedure. If it is successful, you can execute the bin picking routine.
The Trajectory request returns a trajectory as long as there are objects with successfully finished path planning (green). However, do not send the trajectory request several times in a row - this is very dangerous. Always send a scan request first, because the objects in the bin may have moved during the previous trajectory (bin picking routine). In the best case, this causes an unsuccessful pick. In the worst case, it causes a collision.
1.4 Pick-failed request
Required running process
Deployment
Introduced in the Communication interface version
BPS 1.2.0
Required parameters
vision system ID
Description
Tells the vision controller that a pick failed. It increments the pick-failed counter of the object for which the last trajectory/object pose was returned. If this object is again the best pick candidate based on the configured priority settings, it is not chosen. It is chosen only after all objects with successfully planned trajectories and lower pick-failed counters (at the moment the trajectory request is received) have been picked.
For the pick-failed feature to work, set up the pick-failed thresholds correctly. When the object moves a little between scans, it must still be considered the same object (have the same ID). Otherwise, its pick-failed counter is not kept.
It blocks the robot program only for a very short time.
Usage
To use this request, the robot or cell needs a sensor that can detect a failed pick.
When the sensor detects a failed pick, send the request.
1.5 Change scene state request
Required running process
Deployment
Introduced in the Communication interface version
BPS 1.7.0
Required parameters
scene state ID
Description
Requests a change of the active scene state (also called environment state in the Studio).
It blocks the robot program until the requested scene state is loaded successfully or an error occurs.
Usage
Use the request when the collision objects in the robot cell change. The virtual environment must reflect the change so that collision checking works correctly.
1.6 Get vision system status request
Required running process
Deployment
Introduced in the Communication interface version
BPS 1.8.0
Required parameters
vision system ID
Description
Requests the current status of a vision system. The response contains:
It blocks the robot program only for a very short time.
Usage
Use the request whenever you need the status of a vision system.
Returned data
error code,
Info data keys 12 (number of localized objects), 13 (number of ready objects) and 14 (processing state)
1.7 Trigger scan request
Required running process
Deployment
Introduced in the Communication interface version
BPS 1.9.0
Required parameters
vision system ID
TCP (optional - if omitted, the robot pose is taken from the Robot State Server)
Description
The sensor of the requested vision system triggers a new scan.
This request is specific to the Hand-eye MultiView approach, where it is used before the basic Scan request. Hand-eye MultiView must be enabled for the vision system.
The scan can take several seconds. Its result is sent when the scan is finished. To not block the robot program during this time, the response is received in a separate procedure, Wait for scan completion. Always call this procedure after the Trigger scan request.
Usage
The Hand-eye MultiView approach needs scans of the scene from several viewpoints. Send this request at each of these viewpoints (robot poses). At the end, send the basic Scan request. It creates the final virtual scan of the scene and starts the localization and path planning pipeline.
1.8 Localize on the last scan request
Required running process
Deployment
Introduced in the Communication interface version
BPS 1.9.0
Required parameters
vision system ID
Description
This is a variant of the basic Scan request. If a scan from the sensor of this vision system is available, the localization starts to report found objects, and path planning calculates trajectories for them.
The scan does not have to come from the same vision system.
Usage
Use the request in the same way as the basic Scan request, but no new scan is made. Instead, the last scan from the associated sensor is used.
1.9 Object pose request
Required running process
Deployment
Introduced in the Communication interface version
BPS 1.6.0
Required parameters
vision system ID
Description
Requests the pose of the most suitable localized object instead of a bin picking trajectory. The object is chosen in the same way as for the
Trajectory request.
The response contains the object pose in the robot base frame. If the solution provides them, it also contains
Info data: the object dimensions (keys 4-6) and the neural network label (key 7).
It blocks the robot program until the response is received.
Usage
Use the request when the robot program plans the motion to the object itself and only needs to know where the object is. Like the Trajectory request, always send a Scan request for the same vision system first.
Returned data
error code, object pose,
Info data keys 4-7 (when available)
Error codes
No error (0),
Service error (1),
Communication error (3),
Bad data (4),
Timeout (5),
Unknown error (200),
Path planning failed (201),
No object found (202),
Empty scene (218),
Unable to read vision system pose (219),
Wrong bin picking configuration (255)
1.10 Change bounding box request
Required running process
Deployment
Introduced in the Communication interface version
BPS 1.11.0
Required parameters
vision system ID
bounding box ID
Description
Switches the active bounding box of the vision system to the bounding box with the provided ID. A vision system can have several bounding boxes defined in the Studio (also called vision system states). Only the part of the scene inside the active bounding box is used for localization.
It blocks the robot program until the bounding box is switched or an error occurs.
Usage
Use the request when one vision system observes several areas (for example, several bins or several parts of one bin) and the robot needs to pick from a different area. Send it before the Scan request.
2 Locator action requests
2.1 Scan request
Required running process
Deployment
Introduced in the Communication interface version
LS 1.0.0
Required parameters
vision system ID
TCP (only for hand-eye vision system)
Description
The sensor of the requested vision system triggers a new scan (full/fast). Immediately after the scan is finished, the localization starts to report found objects.
For hand-eye vision systems, the request must send the TCP to the vision controller. Use the same TCP and coordinate frame as in the Add calibration point request during calibration. For extrinsic (cell-mounted) vision systems, omit the TCP.
The scan can take several seconds. Its result is sent when the scan is finished. To not block the robot program during this time, the response is received in a separate procedure, Wait for scan completion. Always call this procedure after a scan request.
Usage
Hand-eye vision system
Typically, send it every time you need to localize new objects. Specify the TCP in addition to the vision system ID. The robot arm must be in the scanning position, so that the sensor can see the scene.
Call the Wait for scan completion procedure immediately after the Scan request. It blocks the robot program until the scan is finished (the robot must not move during the scan).
Extrinsic vision systems
Typically, send it every time you need to localize new objects. Omit the TCP.
Then call the Wait for scan completion procedure to receive the result of the scan.
2.2 Get objects request
Required running process
Deployment
Introduced in the Communication interface version
LS 1.0.0
Required parameters
vision system ID
number of requested objects
(timeout - set in the solution settings)
Description
Requests the vision controller to send one or more object poses to the robot controller.
The vision controller chooses the most suitable object according to the sorting criteria. If several objects are requested, they are also sorted.
If the timeout is set to a positive value and several objects are requested, the objects are chosen at the moment the response is sent to the robot controller.
If no suitable object is found, an error is sent to the robot controller.
If the timeout is set to 0, it blocks the robot program only for a very short time (it returns immediately).
If the timeout is set to a positive value, it blocks the robot program until the requested number of objects is found or the timeout is reached.
Usage
Send it when you need a new object pose.
Returned data
error code, object poses, number of returned objects,
Info data keys 4-9 for each object (when available)
Error codes
No error (0),
Service error (1),
Communication error (3),
Bad data (4),
Timeout (5),
Vision system not found (500),
Empty scene (501),
No object found, Detection running (502),
No object found, Detection stopped (503),
All objects rejected, Detection running (504),
All objects rejected, Detection stopped (505)
2.3 Get vision system status request
Required running process
Deployment
Introduced in the Communication interface version
LS 1.1.0
Required parameters
vision system ID
Description
Requests the current status of a vision system. The response contains:
It blocks the robot program only for a very short time.
Usage
Use the request whenever you need the status of a vision system.
Returned data
error code,
Info data keys 12 (number of localized objects), 13 (number of ready objects) and 14 (processing state)
2.4 Trigger scan request
Required running process
Deployment
Introduced in the Communication interface version
LS 1.2.0
Required parameters
vision system ID
TCP (based on usage)
Description
The sensor of the requested vision system triggers a new scan.
This request is specific to the Hand-eye MultiView approach, where it is used before the basic Scan request. Hand-eye MultiView must be enabled for the vision system. If it is not enabled, Wait for scan completion returns
Request forbidden (509).
The scan can take several seconds. Its result is sent when the scan is finished. To not block the robot program during this time, the response is received in a separate procedure, Wait for scan completion. Always call this procedure after the Trigger scan request.
Usage
The Hand-eye MultiView approach needs scans of the scene from several viewpoints. Send this request at each of these viewpoints (robot poses). At the end, send the basic Scan request. It creates the final virtual scan of the scene and starts the localization.
2.5 Localize on the last scan request
Required running process
Deployment
Introduced in the Communication interface version
LS 1.2.0
Required parameters
vision system ID
TCP (only for hand-eye vision system)
Description
This is a variant of the basic Scan request. If a scan from the sensor of this vision system is available, the localization starts to report found objects.
The scan does not have to come from the same vision system.
Usage
Use the request in the same way as the basic Scan request, but no new scan is made. Instead, the last scan from the associated sensor is used.
2.6 Change bounding box request
Required running process
Deployment
Introduced in the Communication interface version
LS 1.4.0
Required parameters
vision system ID
bounding box ID
Description
Switches the active bounding box of the vision system to the bounding box with the provided ID. A vision system can have several bounding boxes defined in the Studio (also called vision system states). Only the part of the scene inside the active bounding box is used for localization.
It blocks the robot program until the bounding box is switched or an error occurs.
Usage
Use the request when one vision system observes several areas and the robot needs objects from a different area. Send it before the Scan request.
3 Calibration action requests
3.1 Add calibration point request
Required running process
Calibration
Introduced in the Communication interface version
BPS 1.6.0
LS 1.0.0
Required parameters
— (BPS - the robot pose is taken from the Robot State Server)
TCP (LS)
Description
The sensor of the requested vision system triggers a scan. Based on the calibration type (extrinsic/hand-eye), the calibration tool (ball/pattern) is localized and a new calibration point is added.
Locator Studio: The request sends the TCP to the vision controller. Use the same TCP and coordinate frame in the scan request as during the calibration.
It blocks the robot program until the calibration point is added successfully or an error occurs.
Usage
Send the request when the robot reaches a calibration pose. The robot must be completely still to get the best scan quality.
3.2 Start automatic calibration request
Required running process
Calibration
Introduced in the Communication interface version
BPS 1.8.0
LS 1.1.0
Required parameters
solution ID
vision system ID
Description
Requests the start of the automatic calibration procedure for the given solution and its vision system.
It blocks the robot program until the calibration process is started successfully or an error occurs.
Usage
Send the request when you need to recalibrate a vision system. For the request to succeed, automatic calibration must be enabled for the vision system, and no other process can be running.
3.3 Save automatic calibration result request
Required running process
Calibration
Introduced in the Communication interface version
BPS 1.8.0
LS 1.1.0
Required parameters
- - -
Description
Requests the vision controller to save the result of the automatic calibration procedure.
It blocks the robot program only for a very short time.
Usage
Send the request after all calibration points are added successfully. It saves the result of the automatic calibration procedure.
Returned data
error code, calibration accuracy (
Info data key 10, in µm), calibration result - the pose of the camera (as a Cartesian pose)
3.4 Stop automatic calibration request
Required running process
Calibration
Introduced in the Communication interface version
BPS 1.8.0
LS 1.1.0
Required parameters
- - -
Description
Requests the vision controller to stop the automatic calibration procedure.
It blocks the robot program until the calibration process is stopped successfully or an error occurs.
Usage
Send the request to stop the automatic calibration procedure.
4 Solution action requests
4.1 Change solution request
Required running process
—
Introduced in the Communication interface version
BPS 1.3.0
LS 1.0.0
Required parameters
solution ID
Description
Requests the vision controller to stop the currently deployed solution and deploy the solution with the provided ID.
For the request to succeed, a solution must already be running in deployment. Also, the requested solution must be ready for deployment (flag Ready), and its robot brand must match the brand of the robot that sent the request.
It blocks the robot program until the deployment of the requested solution is finished, an error occurs, or the timeout (180 seconds) is reached.
Warning: Always make sure that the correct robot model is selected in the requested solution!
Usage
Use the request during deployment when you need to replace the deployed solution with another one.
4.2 Start solution request
Required running process
—
Introduced in the Communication interface version
BPS 1.6.0
LS 1.0.0
Required parameters
solution ID
Description
Requests deployment of the solution with the provided ID.
For the request to succeed, no solution can be running in deployment. Also, the requested solution must be ready for deployment (flag Ready), and its robot brand must match the brand of the robot that sent the request.
It blocks the robot program until the deployment of the requested solution is finished, an error occurs, or the timeout (180 seconds) is reached.
Warning: Always make sure that the correct robot model is selected in the requested solution!
Usage
Use the request to deploy a solution when no other solution is running in deployment.
Error codes
No error (0),
Service error (1),
Communication error (3),
Bad data (4),
Timeout (5),
Bin picking in simulation mode (8),
Invalid solution ID (450),
Another solution running (451),
Deployment not started (452),
Solution is not complete (455),
Invalid robot brand (457),
Solution already running (458),
Solution access unauthorized (459),
No valid license (471)
4.3 Stop solution request
Required running process
—
Introduced in the Communication interface version
BPS 1.6.0
LS 1.0.0
Description
Requests the vision controller to stop the currently deployed solution.
It blocks the robot program until the deployment is stopped. If an error occurs, it blocks only for a very short time.
Usage
Use the request to stop a solution that is running in deployment.
4.4 Get running solution request
Required running process
—
Introduced in the Communication interface version
BPS 1.6.0
LS 1.0.0
Description
Requests the ID of the currently running solution.
A solution must be running in deployment for the request to be successful.
It blocks the robot program only for a very short time.
Usage
Use the request to identify the solution that is running in deployment.
4.5 Get available solutions request
Note:
This action request is not available in all Robot modules.
This action request is deprecated since Communication interface version BPS 1.8.0 and may be removed in a future version. Do not use it in new robot programs.
Required running process
—
Introduced in the Communication interface version
BPS 1.6.0
Description
Requests the list of IDs of the available solutions.
A solution is available if it is ready for deployment (flag Ready). The robot brand selected in the solution does not have to match the brand of the robot that sent the request.
It blocks the robot program only for a very short time.
Usage
Use the request to get the list of all available solutions.
Returned data
error code, solution ID list
5 Response receiving procedures
5.1 Wait for scan completion
Corresponding asynchronous request
Required running process
Deployment
Description
Receives the response with the result of the scan from the vision controller. The vision controller sends this response when the scan is finished.
Always execute it after a Scan request or a Trigger scan request.
It blocks the robot program until the scanning operation has finished and the response is received.
Usage
Hand-eye vision system
Execute the procedure immediately after the Scan request. It blocks the robot program until the scan is finished (the robot must not move during the scan).
Extrinsic vision systems
Usually, execute the procedure when (or just before) the robot is ready to move back into the scanning volume, to make sure that the scan is finished. Also execute it before you send another request. This keeps the time that the scan blocks the robot program to a minimum (usually there is no blocking at all).
5.2 Receive trajectory
Corresponding asynchronous request
Required running process
Deployment
Description
Receives the response from the vision controller. If successful, the response contains the bin picking trajectory and other related data. If not, it contains an error code.
Always execute it after the trajectory request (not necessarily immediately, but before any other request is sent). If it fails, the procedure that executes the bin picking routine also fails.
It blocks the robot program until the response is received. The vision controller sends a successful response (bin picking trajectory with other data) as soon as the first object has a successfully computed trajectory. If several objects have successfully computed trajectories when the trajectory request is received, the most suitable one is chosen based on the configured priority settings.
If an error occurs, the vision controller sends the response:
Immediately
when there is no point cloud (usually when a bounding box is used) -> error No object found
when the empty scene feature is enabled and the scene is evaluated as empty -> error Empty scene
when localization has already finished and no object was found -> error No object found
when localization has already finished and path planning has already failed for all objects -> error Path planning failed
When localization timeout is reached
if it is reached before the trajectory request timeout and no object was found -> error No object found
if it is reached before the trajectory request timeout and path planning for all found objects has finished unsuccessfully before the localization timeout is reached -> error Path planning failed
When path planning for all objects has finished unsuccessfully
When the trajectory request timeout is reached (preceded by the warning “Choosing object to pick timeout” in the Deployment console)
and localization is still running but no object was found yet -> error No object found
and path planning is still running but no object has a successfully computed trajectory yet -> error Path planning failed
Usage
Typically, execute it just before the bin picking routine (while the robot moves to the Start pose or is already there), immediately or shortly after you send the trajectory request.
Returned data
error code, bin picking trajectory, gripper commands,
Info data keys 1-3 (tool point invariance, gripping point ID, gripping point invariance) and, when available, keys 4-7 (object dimensions, neural network label)
Error codes
No error (0),
Service error (1),
Communication error (3),
Bad data (4),
Timeout (5),
Unknown error (200),
Path planning failed (201),
No object found (202),
Vision system not initialized (203),
Unable to read vision system pose (219),
Empty scene (218),
Wrong bin picking configuration (255)
6 Other requests
6.1 Communication check request
Required running process
—
Introduced in the Communication interface version
BPS 1.11.0
LS 1.4.0
Description
Checks that the connection to the Action Request Server is alive. The vision controller answers immediately without executing any action.
It blocks the robot program only for a very short time.
Usage
Use the request to verify the connection, for example after a long pause in production or before the robot starts a new cycle. If it fails, the robot program can reconnect before it sends the next action request.